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ABSTRACT 

The direction of the facilitative effect of questions 
inserted at intervals in prose material is examined in terms of: 1) 

the testual distance of the questions from the material to which it 
refers; and 2) the relationship between the information tested by the 
inserted questions and that tested by the criterion test items. 
Results with 140 undergraduate teacher education students show: 1) 

that the initial effect of inserted questions may be forward, i.e., 
shaping appropriate test inspection behavior; 2) that superior 
performance on pages immediately following questions suggests a 
forward effect mediated through increased attentiveness; and 3) that 
under certain conditions a backward effect is suggested-a 
facilitative review effect in which the facilitation results from a 
memory search initiated by the inserted questions. (Author/TL) 
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ABSTRACT 

THE DIRECTION OF THE EFFECT OF QUESTIONS IN PROSE MATERIAL 



Barry McGaw and Arden Grotelueschen 



Center for Instructional Research and Curriculum Evaluation 

College of Education 

University of Illinois at Urbana-Champaign 



The direction of the facilitative effect of questions inserted at 
intervals in prose material was examined in terms of the textual distance 
of particular information in the passage from the inserted questions and 
the relationship between the information tested by the inserted questions 
and that tested by the criterion test items. The subjects consisted of 
140 undergraduate teacher education students obtained as paid volunteers. 
Results showed that the initial effect of inserted questions may be for- 
ward, shaping appropriate inspection behaviors. In addition, superior 
performance on pages immediately after questions suggested a forward 
effect mediated through increased attentiveness. Superior performance 
on criterion items dealing with the same sect:* of text as the inserted 
questions, but constructed so as to excxuae the possibility of direct 
transfer, suggested a facilitative review effect — the facilitation 
resulting from the memory search initiated by the inserted questions. 



THE DIRECTION OF THE EFFECT OF QUESTIONS IN PROSE nATE^I^L 1 

Barry McGaw and Arden Grotelueschen 

Center for Instructional Research and Curriculum Evaluation 

College of Education 

University of Illinois at Urbana-Champaign 

The insertion of questions in prose material has been shown to 
facilitate learning from the material (Rothkopf, 1966). The effects 
of these questions are both direct, facilitating subsequent performance 
on identical questions, and indirect, facilitating performance on other 
questions about the text materials. The effect is marked when the ques- 
tions refer to preceding material in the text (post-questions) but may 
even be reversed when they refer to subsequent material (p re- quest ions) 
(Frase, 1968; Frase, Patrick and Schumer, 1970; Rothkopf, 1966). 

Although both direct and indirect effects have been reported, the 
operation of these effects has not been clearly explicated. One conjec- 
ture is that the questions serve to shape inspection behaviors thus fac- 
ilitating performance on posttest items dealing with material following 
the questions in the text (Rothkopf, 1963). That is, the effect of the 
questions is forward in that they influence the inspection of materials 
that have not yet been read. From this point of view inspection behav- 
iors are seen to be reinforced (and, thus, maintained) if the inserted 
questions can be answered, or non-re inf orced (and, thus, altered) if the 
inserted questions cannot be answered. A number of studies have strength- 
ened this shaping hypothesis. Rothkopf and Bisbicos ( 1967) observed 
that the facilitation was greater towards the end of the text and that it 
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was selective, being greatest for items in the criterion test which were 
similer to those in the original text. Rothkopf and Coke (1968) found 
that learning was an Increasing function of the likelihood that fragments 
of the text were noticed. Frase (1969), in demonstrating the effects of 
different organizations of prose material, showed that learning was deter- 
mined by the aspects of the text to which the learner could attend. 

Such a forward effect need not operate only through shaping appro- 
priate inspection behaviors, causing Ss to attend to appropriate features 
of the text. It may also function by simply controlling the level of 
attentiveness, causing the reader to attend more carefully to the material 
following each set of inserted questions. The effect of the questions 
would be cyclic, with the effect diminishing as the reader moves through 
the material following the questions but being reinstated following the 
next set of questions. An explanation only in terms of shaping appropriate 
inspection behaviors would predict a cumulative improvement rather than a 
cyclic effect. Although there are no clear data to support this second 
hypothesis there is some suggestive evidence. Rothkopf and Bloom (1970), 
for example, found that reading rate slowed after each set of inserted 
questions. 

The two hypotheses discussed above postulate a forward facilitative 
effect for post-questions. However, the experimental results to date have 
not ruled out the possibility of the indirect facilitative effect of ques- 
tions being a backward, or review, effect. Frase (1968) noted that the 
superiority of post-questions over pre- questions occurred even on the 
first paragraph (though this could be attributed to a suppressive effect 
of pre-questions). Watts and Anderson (1971) found no increase in 
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performance towards the end of their material. Bruning (1968) demonstrated 
that there is an additive review component in the effect of post-questions. 

If the facilitatlve effect of inserted questions is, at least in part, a 
review effect, then the facilitation co'* Id be expected to be greater for 
criterion test items which deal with material related to that reviewed in 
answering the inserted questions. Rothkopf and Bisbicos’ (1967) observa- 
tion of greater facilitation on criterion test items similar to questions 
inserted in the text could be accounted for in this manner. Further, such 
a review effect could be expected to be stronger with short preceding lags, 
that is, with material read shortly before the inserted questions and 
weaker with longer lags. 

If the facilitatlve effect of inserted questions is, at least in part, 
a respondent phenomenon — with the effect of the questions increasing the 
attentiveness to material following the questions — then the effect should 
be greatest with short following lags, that is, on material immediately 
following the questions and weaker with longer lags* 

The purpose of the present study was to test for both forward and 
backward effects of inserted questions, and - determine the conditions 
under which each is most effective. The conditions examined were the 
relationship between the inserted questions and the subsequent questions 
on which facilitation was revealed, and the textual distance of the 
material tested from the inserted questions. 







4 



4 



METHOD 



Materials 



The basic material used in the reported experiments was a selection 
of material from Rachel Carson’s book. The Sea Around Us. The text was 
multilithed onto 21 pages of approximately 260 words each. From each page 
of the material three questions were developed. All 63 questions were of 
the completion type and required the recall of specific information from 
the text. The questions were prepared so that, for each page, two of the 
questions dealt with the same material, while the third dealt with an 
unrelated topic. 

The pairs of questions dealing with the same material were developed 
in such a way that, although they dealt with the same text material, 
neither could be answered from a knowledge of the answer to the other. 

For example, from the following passage in the text a pair of questions 
were formed. 

" *77J the surveying ship uuLudog, examining a 
proposed northern route for a cable from Faroe to 
Labrador in 1860 s came another report. The 
Bulldog ' 8 sounding line 3 which at one place had 
been allowed to lie for some time on the bottom 
at a depth of 1260 fathoms 3 came up wi*$h "IZ star- 
fish clinging to it. 

The two questions developed were: 

(a) The surveying ship which recovered starfish 
from a depth of 1260 fathoms in 1860 3 was 
exploring a route for a cable from Faroe to 



(b) The surveying ship ? which recovered 

starfish from a depth of 1260 fathoms in 1860 3 
was exploring a route for a cable from Faroe . 

The third question developed from the same page as this pair required the 

name of an Arctic explorer. 
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Thus 42 of the 63 questions were matched in pairs. Of these 42, 21 
(one chosen at random from each pair) were selected for insertion in the 
text material for the various experimental treatments. These inserted 
questions are referred to as EQs. The remaining 42 items, 21 matched and 
21 unmatched with the EQs were used to form a criterion test ( Tl ). The 
21 EQs were also used in a criterion test ( T2 ) given after Tl. 

Test Tl was intended to measure the general facilitative effect of 
EQs whereas test T2 was to measure specific learning of the material 
tested by EQs . The two tests were bound into a single test booklet. 

A general measure of reading ability was obtained for all 5s by 
administration of Part II (Reading) of the Reading Comprehension Test 
(Form 1A) from the E.T.S. Cooperative English Tests. 

Treatments 

Three experimental treatments and two control treatments were used. 
All text materials were bound into booklets which were distributed in 
random order, and all 5s took part in the experiments at the same time. 

f 

Because 5s could see one another they v7ere told explicitly, in the printed 
instructions at the front of the booklet, that, although all booklets 
contained the same passages, some were arranged differently from others 
and that they should not be concerned if other 5s appeared to be involved 
in writing, for example, when they were not. The instructions to all 5s 
were to "study each page of the chapter carefully, paying close attention 
to facts and figures and to names and dates." 



Insert Figure 1 about here 
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The treatments, summarized in Figure 1, were as follows: 

El. Experimental Group 1: After the first six pages, i.e., 

after Sections A and B at position ab, a sheet was bound into the text 
booklet with the six EQs from those pages. Similar sheets, each with 
three EQs from the preceding three pages, were bound into the text 
booklet after the ninth, twelfth and fifteenth pages, i.e., at positions 
c, d, and e, respectively. Thus, inserted questions always referred to 
preceding material. 

E2. Experimental Group 2: Questions were inserted in the text 

at positions ab, d, and f in a manner similar to that for El. 

E3. Experimental Group J3: Questions were inserted in the text 

at positions c, e, and g. 

C. Control Group : No insertions were included in the text 

booklets. After the instructions, 5s simply read the 21 pages of text. 

CQ. Contro l Group with Questions : At the same points in the 

text as for the El group, a corresponding number of irrelevant questions 
(CQs) from a Personal Opinion Scale, adapted from a dogmatism scale 
(Rokeach, 1960), were introduced. These questions required approximately 
the same time to complete as the EQ items. The purpose of this treatment 
was to determine the effect of providing break points in the reading 
without text related questions. 
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Subjects 

Subjects for these experiments consisted of 140 undergraduates, 
obtained as paid volunteers, at Concordia Teachers College, River Forest, 
Illinois. Twenty-eight Ss were assigned at random to each of the treat- 
ment groups. 

Procedure 

All Ss attended a single group session. The reading comprehension 
test, requiring 25 minutes, was administered first. The experimental 
booklets were then distributed to Ss, who worked through them at their 
own pace. When a S completed his booklet, he indicated this to a monitor 
who removed it and provided him with a test booklet containing both tests 
T1 and T2. Ss were allowed to leave the room when they had completed 
both tests. The entire pr^ :edure required about 90 minutes for the slower 
Ss. 

RESULTS AND DISCUSSION 

The treatments used in this study form two separate experiments . 
Groups El, C and CQ comprise one experiment (Experiment A) and groups 
E2 and E3 the other (Experiment B) . The results for these are presented 
separately . 

EXPERIMENT A 

Facilitative Effect of Inserted Questions 

Mean scores, raw and adjusted, on the two criterion tests T1 and 
T2 , are shown in Table 1. The adjustment resulted from the use of reading 
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comprehension test score as a covariate. (Test T2 actually contained 21 
items but, for this analysis, only those 15 inserted in the text for 
group El were used./ For test ?1 , the differences among the adjusted means 
were significant, F(2,80) = 3.53, p < .04. 

Insert Table 1 about here 

The difference between the two control groups was not significant. 
Thus, there was no support for the hypothesis that the facilitatlve effect 
of inserted questions was due to increased attentiveness following a rest 
from reading. That is, the effect apparently occurs only with the inser- 
tion of text related questions such as the EQs. 

On test T2 che performance of group El was similarly superior to that 
of the control groups, F(2,80) = 25.86, p < .001. Again the difference 
between the control groups was not significant. Control group CQ was, 
therefore, dropped for subsequent analyses. 

Forward or Backward Effect 

Groups El and C were compared on the 12 items in T1 from sections 
A and B (the first six pages, read before any questions were encountered) 
and the 12 items from sections F and G (read after the last questions had 
been encountered). A forward effect would be expected to produce superi- 
ority of El on the FG items while a backward effect should produce superi- 
ority of El on the AB items. The adjusted means are given in Table 2. 



Insert Table 2 about here 
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The difference between the groups was not significant, F(l,53) = 1.72 
p > .18. Although the overall difference between scores on AB and FG was 
significant, F( 1,54) = 98.15, p < .001, it is of little substantive impor- 
tance since no attempt was made to control for differences in item dif- 
ficulty. The substantively important groups by sections ( AB, FG) inter- 
action was not significant, F(1,5U) = .11. 

There are theoretical grounds, however, for believing that forward 
and backward effects might operate in a more limited fashion than could be 
revealed in such a gross analysis as that shown above. A review effect 
should be greater for criterion items matched with the inserted questions 
and, in particular, for matched items testing material from pages which 
immediately preceded the inserted questions. An attentional increase 

i 

following inserted questions would, on the other hand, exert the greatest 
effect on material from pages immediately following the inserted questions 
an effect which should be revealed by both matched and unmatched items. 

In order to test these hypotheses performance, on all criterion test 
( Tl ) questions dealing with material from pages immediately before and 
immediately after inserted questions was considered. Pages 6, 9, 12 and 
15 were those "before'* insertions and pages 7, .10, 13 and 16 were those 
"after" insertions. From each of these pages there were two questions 
on test Tl, one matched and one unmatched with one of the questions on 
the adjacent insert. Mean performances for the groups are shown in 
Table 3. 

Insert Table 3 about here 



o 

ERLC 



10 



10 



Analysis of covariance (Table 4) revealed group El to have been 
significantly superior to group C in overall performance, F(l,53) = 4.02, 
p < .05. The significant effect for position does not demonstrate that 
performance on items from pages after questions was superior. Since the 
P x G interaction effect was not significant, the position effect was due 
only to differences in item difficulty. The same is true for the signifi- 
cant effects for item type and position x item type interaction. 

Insert Table 4 about here 

The important effect in this analysis is the I x P x G interaction 
effect. Although the test of this effect fell just above the conventional 
level of significance, F(l,54) = 3.74, p < .055, it is examined here 
in some detail since a supplementary analysis of the data in Experiment B, 
reported later, showed the effect to have been replicated there. This 
result indicates that the groups by position of items interaction for 
matched items was significantly different from that for unmatched items. 
The effect is shown in Figure 2, from which the trend in the data can 
clearly be seen. Tests of simple main effects (Winer, 1962, p. 323) 
showed that El was significantly superior to C on matched items from pages 
immediately before inserted questions, F(l,215) = 4.96, p < .05, and on 
unmatched items from pages immediately after inserted questions, 

F(l, 215) « 4.30, p < .05. 

Insert Figure 2 about here 

For the unmatched items there is little likelihood of a review 
effect operating. The results observed with unmatched items can readily 
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be accounted for in terms of a forward effect. After each set of inserted 
questions S s appeared to attend more carefully to the text, hence the supe- 
riority of El over C on items from the first page after each insert. On 
continuing to read the attentiveness to the text presumably diminished 
with a consequent drop in the relative level of performance of El. Such 
a pattern can be seen in the data when it is recognized that pages referred 
to as "before” inserts were three pages after the prior insert. For the 
unmatched items it seems more useful to distinguish the page positions as 
"shortly after" and "long after" rather than "after" and "before." El 
was superior to C on questions from pages shortly after inserts but the 
effect was attenuated with increasing textual distance from the insert — 
the difference between the groups on the "long after" pages being insig- 
nificant. 

Such a forward effect should also operate with the matched items but, 
with these items, there is the additional possibility of a review effect 
facilitating performance on material from pages prior to the inserts. The 
data for the matched items suggest that such a review effect did, in fact, 
operate. El was significantly superior to C on matched items from pages 
immediately prior to inserts but not on pages after the inserts. These 
"after" pages, for consideration of a review effect, are better referred 
to as "long before” inserts. Just as the facilitative forward effect on 
the unmatched items was attenuated with increased textual distance after 
the insert, the facilitative review effect on matched items was attenuated 
with increased textual distance before the inserts. 
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EXPERIMENT B 

The experiment with groups E2 and E3 was designed to provide a further 
test of the alternative forward and backward hypotheses. The design of this 
experiment can be seen in Figure 1. For group E2 inserted questions occurred 
before Sections C, E, and G whereas, for group E3 they occurred after these 
sections . 

Forward or Backward Effect 

The adjusted mean performances of the two groups, on the items in test 
11 which were drawn from the pages in C, 2, and G, are shown in Table 5. 

The data from whicli these means were obtained were analyzed is a repeated 
measures ANOVA with covariance adjustments on the between subjects variable. 
This analysis showed that the overall difference between the groups was not 
significant, F(l,53) - .013. The groups by sections interaction effect, 
however, was significant, F(2»108) = 6.81, p < .002. 

Insert Table 5 about here 

Tests of the simple main effects (Winer, 1962, p. 311) for groups, 
for each section of the material, revealed that the superiority of E2 on 
Section C approached significance, F(l,161) = 3.1, p < .10, that the 
superiority of E3 on Section E was significant, F(l,161) = 4.4, p < .05, 
and that there was no significant difference between the groups on 
Section G. 

These data provide important information about the nature of the 
forward and backward effects. On Section C, prior to which only group E2 
had received inserted questions, the performance of E2 was superior. The 




13 



13 



Inspection behaviors of Ss in E2 had apparently been shaped, on seeing the 
earlier questions, so that they attended to the specific factual infor- 
mation tested by the inserted questions and the criterion test. The extent 
to which inserted questions can have a facilitative shaping effect depends 
on the extent to which 5s hab .ud inspection behaviors are inappropriate 
for the particular text material. Whether the effect would be less marked 
with questions of a different tyi>e from those used in the present study 
largely remains to be shown. Watts; and A' derson (1971), in fact, reported 
data suggestive of a review effect father than a shaping effect with 
"application" questions. 

Prior to Section E, on which tisse performance of E3 was superior, both 
groups had encountered inserted questions,. This superiority was obtained 
despite the fact that group E2 had encountered two sets of inserted ques- 
tions, including one immediately prior to the section. The superiority of 
E3 can, therefore, be attributed to a review effect, occurring because of 
the inserted questions immediately after the section. Thus, it appears 
that inserted questions may serve a review function only if appropriate 
inspection behaviors have been used. 

The non-significant difference on the final section G cannot be 
accounted for but an explanation might be due to a recency effect oblit- 
erating the advantage to E3 of having questions at the end of the text 
immediately prior to taking the criterion test. 

Replication of Results of Experiment A 

The primary analysis for Experiment B, reported above, indicated 
that both shaping and review effesets occur. The data in Experiment A 
suggested that the review effect wras greatest with matched items. A 
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further analysis was made of the data from Experiment B to determine whether 
the effect on matched and unmatched questions had been replicated. In order 
to do this the data from control group C in Experiment A was included. 

For group E2 the pages immediately before * 'serts were 6, 12 and 18 
and those after were 7, 13, and 19. These constitute page group A in 
Table 6 in which the adjusted mean performanc of the groups are shown. 

Page group B is the set of pages before (9, 15, 21) and after (10, 16) for 
group E3. 

Insert Table 6 about here 

The analysis for groups E2 and C, with data from page group A, 
produced precisely the results noted in Experiment A, viz a significant 
groups x page position x item type interaction, F(l,54) = 6.07, p < .02. 

The graph of this interaction, in Figure 3, shows the effect to be the 
same as before, superior performance for the group with Inserted questions 
on matched items from pages immediately prior to inserted questions, 

F(l,215) = 2.1, p < .15, and on unmatched items immediately after inserted 
questions, F(l,215) = 4.4, p < .05. 

Insert Figure 3 about here 

The analysis for groups E3 and C revealed a non-significant groups x 
page position x item type interaction, F(l,54) = .07, but, in this case, 
the groups x item type interaction was significant, F(l,54) = 3.94, p < .05. 
Group E3 was significantly superior on matched items regardless of distance 
from the insert but there was no significant difference on unmatched items. 
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CONCLUSIONS 

Previous research on the effect of Inserted questions has shown both 
direct and indirect facilitation of subsequent performance on questions 
relating to the text. The indirect facilitation has occurred on items to 
which, it had been demonstrated, there was no direct transfer model in 
which knowledge of the answer to one question facilitates answering 
another. In this study the matched questions were constructed In such a 
way that the answer to one could not in any way provide the answer to the 
other member of the pair. Yet review of the material required to answer 
an inserted question facilitated performance on its matched item, provided 
that the review occurred within a page or so of the relevant material. 

The nature of the retrieval phenomenon, or memory search, which gives 
rise to the facilitation is not yet clear. The important variable could 
be similarity of subject matter though, in this study, the similarity 
occurred only in the subject matter of the sentence from which the re- 
sponses were deleted, not in the responses themselves. Alternatively the 
important variable could be proximity of material in the text, with 
greater facilitation for contiguous material. A further possibility Is 
that verbatim recall of the original sentence to complete the inserted 
question provided also the word required to complete the matched question, 
despite the fact it was also deleted from the inserted question. Not all 
of the questions required simple verbatim recall but further research with 
controlled use of verbatim and paraphrase items will clarify this issue. 

This study also confirmed the existence of a forward effect on the 
unmatched items. The results of Experiment B provide support for a 
shaping hypothesis, suggesting that, insofar as Ss habitual inspection 
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behaviors are inappropriate, inserted questions will ,..rve t<*> shape 
appropriate behaviors. The use of questions testing ighly specific 
factual information probably highlighted this effect. The results of 
Experiment A suggested that, in addition to a shaping effect, the inserted 
questions serve also to control general attention^ behaviors. 5s per- 
formed better on material from pages immediately afteir the inserts. 

Such control of attention, however,. was only achieved with text related 
questions. 
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TABLE 1 

Mean Scores on Criterion Tests by Groups 
for Experiment A 





T1 




T2 




Group 


Raw Mean 


Adjusted Mean 


Raw Mean 


Adjusted Mean 


El 


15.7 


16.0 


8.14 


8.27 


C 


13.5 


13.5 


4.25 


4.23 


CQ 


14.1 


13.8 


4.64 


4.52 
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TABLE 2 

Adjusted Means on AB and FG Items in Test T1 
by Groups for Experiment A 



Groups 







Items 






AB 


FG 


Overall 


El 


2.05 


4.86 


3.46 


C 


1.80 


4.31 


3.05 


Overall 


1.93 


4.59 


3.25 




22 



22 



TABLE 3 

Adjusted Means on Matched and Unmatched Items 
Before and After Inserts by Groups for Experiment A 



Matched Unmatched 



Groups 


Before 


After 


Before 


After 


El 


1.59 


2.05 


1.95 


2.41 


C 


1.01 


1.87 


1.79 


1.87 


Overall 


1.30 


1.96 


1.87 


2 . 14 
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TABLE 4 

Analysis of Covariance for Matched and Unmatched Items 
Before and After Position of Inserts 
for Experiment A 



Source 


SS 


df 


MS 


F 


Between subjects 


Groups [G] 


7.44 


1 


7.44 


4.02 


Subjects within groups [S(G)] 


98.05 


53 


1.85 




Within subjects 


Position of items [P] 


12.07 


1 


12.07 


14.67 


PxG 


.00 


1 


.00 


.00 


PxS(G) 


44.42 


54 


.82 




Type of item [I] 


7.87 


1 


7.87 


13.89 


IxG 


.01 


1 


.01 


.03 


IxS(G) 


30.60 


54 


.56 




IxP 


2.16 


1 


2.16 


3.74 


IxPxG 


2.16 


1 


2.16 


3.74 


IxPxS(G) 


31.17 


54 


.57 
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p 



.047 



.000 



.001 

.854 

.055 

.055 
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TABLE 5 



Adjusted Means 


on Sections C, E, and 


G by Groups 


for 


Experiment B 


Group 


Section C 


Section 


E 


Section G 


E2 


2.27 


3.22 




2.33 


E3 


1.75 


3.84 




2.16 



O 

ERIC 



25 



25 



TABLE 6 

Adjusted Means on Matched and Unmatched Items 
Before and After Inserts by Groups for Experiment B 







Before 


After 






Matched 


Unmatched 


Matched Unmatched 


Page Group A 


E2 


1.40 


1.31 


1.38 1.63 


(6,7,12,13, 










18,19) 


C 


1.09 


1.33 


1.43 1.18 



Page Group B 
(9,10,15, 


E3 


1.28 


1.20 


.72 


.90 


16,21) 


C 


.92 


1.43 


.44 


.95 
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FIGURE CAPTIONS 



Fig. 1. Text booklet sequence for different treatment groups in 
Experiments A and B. 

Fig. 2. Groups by position of item interaction for different types 
of items for Experiment A. 

Fig. 3. Groups by position of item interaction for different types 
of items for Experiment B. 
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MEAN NUMBER OF ITEMS CORRECT 



28 




El 




1 
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BEFORE 



MATCHED ITEMS 



UNMATCHED ITEMS 
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MEAN NUMBER OF ITEMS CORRECT 
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MATCHED ITEMS UNMATCHED ITEMS 
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